Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[FEATURE] Add feature of attach_grad to nonleaf variables in HybridizedBlock.#21091

Open
KexinFeng wants to merge 3 commits intoapache:masterfrom
KexinFeng:gluon_rg_branch
Open

[FEATURE] Add feature of attach_grad to nonleaf variables in HybridizedBlock.#21091
KexinFeng wants to merge 3 commits intoapache:masterfrom
KexinFeng:gluon_rg_branch

Conversation

@KexinFeng
Copy link
Contributor

@KexinFeng KexinFeng commented Jul 8, 2022 •
edited
Loading

Description

This is the same as this #20559. The PR adds the support for fetching the gradients of intermediate variables in a gluon HybridizedBlock. This applies uniformly to both when block.hybridize() is on and off. This generates the attach_grad implemented in implemented in PR#20500.

The motivation of this feature comes from this issue#11865.

Checklist

Essentials

  • PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Changes

  • block.py where mark_vars and get_mark_vars are added along with MXNDArrayMarkDCVariables.
  • cached_op.invoke in cpp backend and CachedOp.__call__ have been editted to include the pass of marked nonleaf ndarrays.
  • set_nleafs method is added into CachedOp class to store the marked nonleaf ndarrays.
  • Inside void RunGraph, marked nonleaf ndarrays are linked to the marked computational node for autograd computation.

Comments

  • This feature is built on top of PR#20500. The modification here is mainly in the invoke of CachedOp computation.

@mxnet-bot
Copy link

mxnet-bot commented Jul 8, 2022

Hey @KexinFeng , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [website, clang, miscellaneous, sanity, windows-cpu, edge, windows-gpu, centos-cpu, unix-cpu, unix-gpu, centos-gpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Jul 8, 2022
@KexinFeng
Copy link
Contributor Author

KexinFeng commented Jul 8, 2022

@mxnet-bot run ci [unix-cpu, unix-gpu, windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 8, 2022
@mxnet-bot
Copy link

mxnet-bot commented Jul 8, 2022

Jenkins CI successfully triggered : [windows-gpu, unix-cpu, unix-gpu]

@mseth10 mseth10 added pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress labels Jul 8, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 8, 2022
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 8, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 8, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 8, 2022
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 10, 2022
@mxnet-bot
Copy link

mxnet-bot commented Jul 11, 2022

Jenkins CI successfully triggered : [centos-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 11, 2022
@KexinFeng
Copy link
Contributor Author

KexinFeng commented Jul 11, 2022

@mxnet-bot run ci [macosx-x86_64]

@mxnet-bot
Copy link

mxnet-bot commented Jul 11, 2022

None of the jobs entered are supported.
Jobs entered by user: [macosx-x86_64]
CI supported Jobs: [sanity, centos-cpu, clang, unix-cpu, windows-cpu, windows-gpu, miscellaneous, centos-gpu, unix-gpu, edge, website]

@KexinFeng
Copy link
Contributor Author

KexinFeng commented Jul 11, 2022

@mxnet-bot run ci [macosx-x86_64]

@KexinFeng KexinFeng closed this Jul 11, 2022
@mxnet-bot
Copy link

mxnet-bot commented Jul 11, 2022

None of the jobs entered are supported.
Jobs entered by user: [macosx-x86_64]
CI supported Jobs: [sanity, centos-cpu, clang, unix-cpu, windows-cpu, windows-gpu, miscellaneous, centos-gpu, unix-gpu, edge, website]

@KexinFeng KexinFeng reopened this Jul 11, 2022
@KexinFeng KexinFeng closed this Jul 11, 2022
@KexinFeng KexinFeng reopened this Jul 11, 2022
@KexinFeng KexinFeng closed this Jul 12, 2022
@KexinFeng KexinFeng reopened this Jul 12, 2022
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Aug 31, 2022
@KexinFeng
Copy link
Contributor Author

KexinFeng commented Sep 22, 2022

@mxnet-bot run ci [edge, unix-gpu]

@mxnet-bot
Copy link

mxnet-bot commented Sep 22, 2022

Jenkins CI successfully triggered : [unix-gpu, edge]

@KexinFeng
Copy link
Contributor Author

KexinFeng commented Oct 8, 2022

@mxnet-bot run ci [unix-gpu]

@mxnet-bot
Copy link

mxnet-bot commented Oct 8, 2022

Jenkins CI successfully triggered : [unix-gpu]

@KexinFeng
Copy link
Contributor Author

KexinFeng commented Oct 11, 2022

@mxnet-bot run ci [unix-gpu]

@mxnet-bot
Copy link

mxnet-bot commented Oct 11, 2022

Jenkins CI successfully triggered : [unix-gpu]

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@eric-haibin-lin eric-haibin-lin Awaiting requested review from eric-haibin-lin eric-haibin-lin is a code owner

@szha szha Awaiting requested review from szha szha is a code owner

Assignees

No one assigned

Labels

pr-awaiting-testing PR is reviewed and waiting CI build and test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@KexinFeng @mxnet-bot @mseth10